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LEADING 

Not Lagging 



BY ROBERT D. SOMMERS 

A t Ohio’s Butler Tech, stu- 
dent success is our business. 
Student success, as measured 
by student results, is how we 
evaluate our programs and services. But 
we don’t focus on these evaluative data 
sets on a daily basis; we focus on a series 
of quality measures that are highly 
correlated with student success. These 
measures, called leading measures or 
process performance measures, are col- 
lected and analyzed on a regular basis at 
all levels of the organization. They serve 
as constant feedback on how well we are 
succeeding with students. These non- 
evaluative measures guide our daily work 
and help keep us on track for the end-of- 
program or end-of-high-school evaluation 
measures that we value. They inform our 
process management and assure work 
quality. 

Getting Started 

We created our leading measures through 
a process depicted in hgure A. Our work 
was facilitated by Robert Sheets at the 
University of Illinois at Urbana- 
Champaign. 

1 . We clarihed our dehnition of student 
success, which varies based on pro- 
gram purpose. Our alternative schools 
have a purpose different from that of 
our career development program. 


and these programs are different from 
our family and consumer science, on- 
line, customized training, and work- 
force programs. Dehning what student 
success means is a critical hrst step 
in the process of making improve- 
ments. For Perkins-supported career 
and technical programs the dehnition 
was successful transition to work and 
higher education. 

2. We developed a set of quantitative 
performance measures to determine 
the extent to which our students were 
being successful within each program. 
For Perkins-supported programs we 
used the performance measures out- 
lined in the federal Perkins legislation. 

3. With the performance measures 
clearly dehned, we determined what 
inhuenced student success. These 
inhuencers were separated into two 
sets; those we could control and those 
we could not. 

The inhuencers we couldn’t control 
such as family circumstances, poverty 
and prior academic achievement were 
set aside. Peter M. Senge notes a learn- 
ing disability titled “The enemy is out 
there” in his book The Fifth Discipline. 

He describes how blaming something or 
someone outside of our control for our 
current situation lets us avoid learning 
and growth. It relieves us of responsibility 
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Leading measures, or process performance measures, are collected 
and analyzed on a regular basis at all levels of the organization. 
They serve as constant feedback on how well we are succeeding 
with students. These non-evaluative measures guide our daily work 
and help keep us on track for the end-of-program evaluation. 
They inform our process management and assure werk quality. 


for our circumstances. We were deter- 
mined to avoid this learning disability. 
Not blaming student performance on 
inhuencers we couldn’t control meant we 
had to focus on those inhuencers within 
our control. We had to take responsibility 
for student success! 

4. The inhuencers we controlled were 
grouped and a series of major process- 
es were identihed. These major pro- 
cesses included: 

a. Governance 

b. Curriculum design and 
development 

c. Education delivery 

d. Administrative services 

e. Buildings and grounds 

f. Enrollment, retention and 
transition 

g. Einance and accounting 

h. Human resources management 

i. Information management 

j. Eeadership 

k. Marketing and public relations 

l. Security and safety 

m. Student support services 

n. Technology 

5. A set of key requirements were estab- 
lished for each of the major processes. 
These key requirements (see sidebar 
example Educational Delivery (hgure 
B)) described the conditions that 
needed to exist to properly control 
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the inhuencers that would lead to 
student success. 

6. The key requirements drove the 
creation of a series of procedures we 
named administrative guidelines. 
These were daily approaches to our 
major processes. These administrative 
guidelines were assigned an owner 
and a cross-functional team. 

7. A process performance measure was 
created for each administrative 
guideline. These leading measures 
were selected to provide feedback on 
the extent to which the procedures 
we created were being effectively 
implemented. A select group of these 
leading measures include those listed 
in hgure C. These measures are 
highly correlated with student 
success — the ultimate evaluation of 
our work. Over time they may change 
as we rehne the cause and effect 
relationship between these leading 
measures and student performance. 

This process of creating leading measures 
is critical to the long-term acceptance 
and success of any monitoring system. 
Although it may be easy to take the work 
we did at Butler Tech and simply imple- 
ment it in another school district, it will 
ultimately fail. Eaculty and staff have 
to wrestle with the difficult discussions 
surrounding student success, inhuenc- 
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ers, major processes, key requirements, 
and leading measures before they can be 
ready to use these tools to create change. 
Although some of the work may be the 
same, each organization will have its own 
unique response to these important areas. 

Putting the Measures to Work 

The leading measures are only effective 
if they are monitored and analyzed. This 
is a daunting ehallenge when day-to-day 
activities demand so mueh attention in 
most districts. To assure monitoring and 
analysis, the administrative guideline 
owners and their teams are eharged with 
the task of assuring that leading measures 
are eollected, analyzed and reported to 
senior leadership. Senior leadership re- 
inforees the analysis of the measures. The 
combination of a series of cross-functional 
teams monitoring and analyzing results 
and the regular organizational strueture 
supporting deep analysis dramatically 
improves the quality of analysis and 
attention. 

An example of a leading measure is 
student engagement in standards-aligned 
currieulum work. The researeh is very 
clear: engage students in active learning 
that is aligned to aeademic and industry 
standards and they will succeed in post- 
seeondary pursuits. We monitor student 
engagement using the walk-through 
proeess outlined in the book The Three 
Minute Classroom Walk-Through (Downey, 
Steffy, English, Erase, Poston, 2004). Ad- 
ministrators monitor student engagement. 


eurriculum alignment, and other critieal 
factors on a biweekly basis in every elass- 
room in the district. This information is 
provided as professional, non- evaluative 
feedback to faculty so they ean adjust 
instruetion to assure full student engage- 
ment. It is also colleeted at the building 
and the distriet level. It drives decisions 
related to elassroom praetice, technol- 
ogy usage and support, extra help and 
intervention services, and professional de- 
velopment. Administrator time to monitor 
student engagement was made possible by 
the elimination of faculty evaluations for 
faeulty who produce strong student 
performance. 

The student engagement measure is 
assigned to the Currieulum Delivery 
Supervision administrative guideline. 

This guideline’s team is able to assess the 
extent to whieh engagement is oeeurring 
by monitoring reports produced by the 
Butler Teeh data warehouse. Division, 
building and faeulty-level data are avail- 
able to everyone, in addition to organiza- 
tional-level analysis. The faculty receives 
this information on a biweekly basis. 

They, of course, can also monitor student 
engagement directly on a daily basis. 

Continuous Improvement... 
a Challenge in New Thinking 

The leading measures are only useful 
if they affect daily work. Core to this 
ehange are the administrative guideline 
teams and affeeted faeulty and staff. 

The guideline teams analyze the leading 
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Figure B: Education Delivery 


Key requirements for effective 
education delivery: 

1 . Education is delivered as planned with 
faculty having access to a course of study 
and instructional guides appropriate to their 
assignment. 

2. Faculty will have the ability to use the 
appropriate methodology and differentiated 
instruction to maximize student engagement. 

3. Efficient delivery of courses of study produces 
student performance. 

4. Students receive high-quality instruction 
consistent with the Butler Tech Learning 
Experience criteria, including: 

a. Students are engaged in active learning. 

b. Students are given extra help when 
needed. 

c. Students are given regular ongoing 
feedback on performance against content 
standards. 

d. Students are prepared to take external 
assessment (/.e., industry credentials, ACT, 
Ohio graduation exam). 

e. Students are treated with respect. 

f. Students find learning enjoyable and 
challenging. 


measures to learn what works and what 
doesn’t. Butler Teeh employees at all 
levels engage in best practice research, 
professional development, and experimen- 
tation. The guideline teams listen to the 
feedback provided from these employees, 
then combine their learning and listening 
into recommendations for upgrading the 
procedures outlined in the administrative 
guidelines. This continuous improvement 
is critical to our success and our sustain- 
ability. 

One of our greatest challenges is 
shifting our thinking from the evaluation 
to feedback. It is quite painful the hrst few 
times you look at leading measures that 
are far from perfect. The tendency is to 
want to only report what we do well. But 
continuous improvement requires you to 
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g. Students are given confidence that they 
can succeed and believe effort will 
produce results. 

h. Students have the necessary time to 
learn what is assessed. 

i. Students know what is expected and 
what will be assessed. 

j. Students recognize the importance 
of career and educational plans and 
career passports. 

k. Students recognize the relevance of 
student organization membership and 
activities to their career and educational 
goals. 

l. Students recognize the relevance of what 
they are learning to their career and 
educational goal. 

m. Students take personal responsibility 
for learning. 

n. Student achievement is recognized 
and rewarded. 

0 . Students have at least one staff or faculty 
member who takes a holistic interest 
in them. 

p. Students have access to equipment and 
supplies necessary to prepare them for 
success as defined by the curriculum. 


seek out failure and to be exeited to hud 
it. Binding failure points, and going about 
eorreeting the proeedures that produee 
the failure, is hard work. Aeeepting our 
shorteomings is difheult. The extent to 
whieh we ean eelebrate bad news and hnd 
it valuable is direetly related to the extent 
to whieh eontinuous improvement ean 
oeeur. 

Critieal to the open, honest analysis 
proeess is the prohibition of this informa- 
tion being used in faeulty or staff evalua- 
tions. All leading measures are non- 
evaluative in nature. This allows everyone 
to openly diseuss the result, seek assis- 
tanee freely, and take risks with ereating 
new ways to improve results. One of our 
eore beliefs is that anything we do today 
is inherently flawed and must be replaeed 




Figure C: Leading Measures 


1 . Alignment of board actions to board policy. 

2. Policy quality. 

3. Quality of curriculum as determined by 
PDK criteria. 

4. Student engagement in instruction aligned 
to content in an approved course of study. 

5. Number of building safety code violations. 

6. Percent of program enrollment capacity 
achieved. 

7. Budget within board budget parameters. 

8. Percentage of positions filled with qualified 
personnel as determined through 

an annual audit. 

9. Percentage of required data in official 
data systems. 

10. Students with documented personal and 
social barriers receiving necessary 
support services. 


Our approach of always 
coming back to student success 
as the only measure of 
excellence keeps the leading 
measures a useful feedback 
system for improving processes. 
It also assures we don't forget 
what is most important — 
our students. 


with a better approaeh. Anyone who looks 
baek on their work with satisfaetion is 
either a fool or someone who hasn’t 
learned anything. Ellen Glasgow onee 
said, “No idea is so antiquated that it was 
not onee modern. No idea is so modern 
that it will not someday be antiquated.” 

By foeusing on improving leading mea- 
sures instead of pushing the latest reform 
initiative, we are able to deeply imbed 
eontinuous improvement praetiees. 

Our Results 

Butler Teeh has been able to improve its 
student performanee results from near 
the worst in Ohio to among the best in 
six years. This dramatie improvement in 
student sueeess oeeurred at the same time 
the distriet doubled its full-time equiva- 
lent student enrollment from 1,615 to 
nearly 3,400. The distriet eontinues to be 
one of the lowest eost (per pupil) eareer- 
teehnieal distriets in Ohio as well. All of 
the hard work is paying off: The institu- 
tion earned the prestigious Aehievement 
of Exeellenee Award from Ohio’s Baldrige 
Quality program for its work in organi- 


zational quality and student performanee 
improvements. We are eurrently using 
the National Baldrige Quality Award 
program as a feedbaek system to further 
improve our proeesses and our 
performanee. 


Staying Focused on Student Success 

We evaluate our work by the number of 
students who sueeeed. We assure student 
sueeess by using leading measures to 
monitor the quality of our work. Our 
approaeh of always eoming baek to stu- 
dent sueeess as the only measure of exeel- 
lenee keeps the leading measures a useful 
feedbaek system for improving proeesses. 
It also assures we don’t forget what is most 
important — our students. D 


Robert D. Sommers, 


is the CEO of Butler Technology and Career Development 
Schools (www.butlertech.org) in southwestern Ohio. 

He can be reached atsommersb@butlertech.org. 




Interested in exploring this topic 
further? Discuss it with your 
colleagues on the ACTE forums at www. 
acteonline.org/forum.aspx. 
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STATES DEVELOP 

QUALITY DATA SYSTEMS 


BY JASON KIKER 

Q uality data is a phrase that 
has quickly become part of 
administrators’, teachers’ 
and policymakers’ vocabu- 
lary. With federal and state legislation 
increasing the amount and types of data 
that schools, districts and states have to 
collect, new questions are being asked 
that include what is the best data to use 
for improving student performance in 
secondary and postsecondary education; 
how to link data so students can be fol- 
lowed from secondary to postsecondary 
and into the workforce; how are privacy 


concerns of students and parents met; 
how can information best be shared; and 
what impact can data have on high-stakes 
testing and other skill assessments. 

While there is no doubt that data 
systems can improve the managment of 
secondary and postsecondary education 
systems and are necessary for other state 
reporting requirements, the potential uses 
of these systems — from linking students’ 
course-taking patterns to remedia- 
tion enrollment in college, identifying 
best practices in classroom instruction 
and tracking students’ success in the 


workplace — far surpass the basic state 
and federal reporting requirements that 
inform much of a state’s use of P-20 edu- 
cation data. 

Quality data and data systems are also 
important to states when trying to decide 
which educational programs to fund. Be- 
ing able to see the effectiveness of a career 
and technical education (CTE)/academic 
skills integration program or a class size 
reduction pro-gram means the state will 
not continue to spend money on programs 
that are not helping students, teachers or 
schools improve. 
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